Crystal Loss and Quality Pooling for Unconstrained Face Verification and Recognition

In recent years, the performance of face verification and recognition systems based on deep convolutional neural networks (DCNNs) has significantly improved. A typical pipeline for face verification includes training a deep network for subject classification with softmax loss, using the penultimate layer output as the feature descriptor, and generating a cosine similarity score given a pair of face images or videos. The softmax loss function does not optimize the features to have higher similarity score for positive pairs and lower similarity score for negative pairs, which leads to a performance gap. In this paper, we propose a new loss function, called Crystal Loss, that restricts the features to lie on a hypersphere of a fixed radius. The loss can be easily implemented using existing deep learning frameworks. We show that integrating this simple step in the training pipeline significantly improves the performance of face verification and recognition systems. Additionally, we focus on the problem of video-based face verification, where the algorithm needs to determine whether a pair of image-sets or videos belong to the same person or not. A compact feature representation is required for every video or image-set, in order to compute the similarity scores. Classical approaches tackle this problem by simply averaging the features extracted from each image/frame of the image-set/video. However, this may lead to sub-optimal feature representations since both good and poor quality faces are weighted equally. To this end, we propose Quality Pooling, which weighs the features based on input face quality. We show that face detection scores can be used as measures of face quality. We also propose Quality Attenuation, which rescales the verification score based on the face quality of a given verification pair. We achieve state-of-the-art performance for face verification and recognition on challenging LFW, IJB-A, IJB-B and IJB-C datasets over a large range of false alarm rates (10−1 to 10−7).